P4Hauler: An Accelerator-Aware In-Network Load Balancer for Applications Performance Boosting

Abstract

Programmable accelerators enable the execution of applications intended for running in usual servers. However, inappropriately running applications on these devices can lead to load imbalance and performance degradation. An alternative to tackle this problem is load balancing, but existing in-network load balancers typically have no visibility of accelerators and often hard code policies in the switch source code. In this article, we present P4Hauler , an accelerator-aware in-network load balancer. In particular, our design discusses how to enforce load-balancing decisions in a programmable switch in a resource-aware manner, allowing different policies to handle traffic according to applications’ needs. We use monitoring and compression techniques to store application resources in a programmable switch for resource-aware decisions. In addition, we propose building blocks that operators can dynamically choose to realize different load balancing policies on-the-fly. We implemented and evaluated a prototype of P4Hauler on a testbed to show its efficiency and deployment feasibility. Our results indicate that P4Hauler can support 27% more load and decrease the flow completion time by around 13% using only a single accelerator. Also, extensive simulations confirm the performance gain of P4Hauler at scale compared to the state-of-the-art.