Background traffic optimization for meeting deadlines in data center storage

Abstract

Background traffic, such as repair, rebalance, backup and recovery traffic, often has large volume and consumes significant network resources in cloud storage systems. While having each application independently schedule its own background traffic can easily generate interference among data flows, causing violation of desired QoS requirements (e.g., latency and deadline), heuristic scheduling algorithms like Earliest-Deadline-First and First-In-First-Out are not able to take into account data center constraints such network topology or data chunk placement, thus resulting in unsatisfactory performance. In this paper, we propose a new algorithm, Linear Programming for Selected Tasks (LPST), which coordinate background traffic of different jobs to meet traffic deadline and optimize system throughput. In particular, our goal is to maximize the number of background traffic flows that meet their target deadlines under bandwidth constraints in data center storage systems. Using realistic traffic trace, our simulation results show that the proposed algorithm significantly improves task processing time and the probability of meeting deadlines.