Overcome Heterogeneity Impact in Modeled Fork-Join Queuing Networks for Tail Prediction

Inspired by the potential power of random scheduling at data centers, a novel approach for combining arbitrary dispatching policy and tail-latency prediction in heterogeneous fork-join network environments is proposed. Tail prediction is of practical importance in commercial data centers, where the need for sharing resources between many applications is desired at most, to ensure client satisfaction with guaranteed service level objectives (SLOs). Lots of research works in parallel scheduling were presented using event-based simulations, but none of them were able to implant dynamic variation of tasks numbers and maintain the determined load region using a precise, and reliable approach. In this paper, we propose extensive case studies for the presented prediction model in heterogeneous black-box using model-driven simulations. Experimental results show that by using random scheduling algorithm accompanied with inserted effects of different requests fan-out, tail latency can be predicted and stay consistent with relative errors of 10% at high load regions.