On the performance evaluation of distributed join-idle-queue load balancing with and without token withdrawals

Distributed Join-Idle-Queue load balancing is known to achieve vanishing waiting times in the large-scale limit provided that the number of dispatchers remains fixed, while the number of servers tends to infinity. When the number of dispatchers $m$ scales to infinity together with the number of servers $n$, such that $r=n/m$ remains fixed, the large-scale performance of Join-Idle-Queue load balancing is less clear as waiting times no longer vanish.

In this paper we first discuss some existing mean field models for distributed Join-Idle-Queue load balancing with $r=n/m$ fixed and explain why the well-known model introduced in Lu et al. (2011) is not exact in the large-scale limit. The inexactness is caused by mixing two variants of distributed Join-Idle-Queue load balancing: a variant with and one without token withdrawals. Next we introduce mean field models for Join-Idle-Queue load balancing with and without token withdrawals, where an idle server places a token at a dispatcher with the shortest among $d$ randomly chosen dispatchers.

The introduced mean field models in case of token withdrawals imply that for phase type distributed service times and a total job arrival rate of $\lambda n<n$, the response time of a job corresponds to that in a standard M/PH/1 queue with load $\lambda q_0$. The value of $q_0$ can be determined numerically and depends on $\lambda ,r$ and $d$, but not on the job size distribution (apart from its mean). This simple behavior is lost if token withdrawals do not take place. For the models without withdrawals we develop fast numerical algorithms to determine the performance. We present simulation experiments that suggest that the unique fixed point of the introduced mean field models provides exact results in the large-scale limit.