Resource Auto-Scaling and Sparse Content Replication for Video Storage Systems

Abstract

Many video-on-demand (VoD) providers are relying on public cloud providers for video storage, access, and streaming services. In this article, we investigate how a VoD provider may make optimal bandwidth reservations from a cloud service provider to guarantee the streaming performance while paying for the bandwidth, storage, and transfer costs. We propose a predictive resource auto-scaling system that dynamically books the minimum amount of bandwidth resources from multiple servers in a cloud storage system to allow the VoD provider to match its short-term demand projections. We exploit the anti-correlation between the demands of different videos for statistical multiplexing to hedge the risk of under-provisioning. The optimal load direction from video channels to cloud servers without replication constraints is derived with provable performance. We further study the joint load direction and sparse content placement problem that aims to reduce bandwidth reservation cost under sparse content replication requirements. We propose several algorithms, and especially an iterative L1-norm penalized optimization procedure, to efficiently solve the problem while effectively limiting the video migration overhead. The proposed system is backed up by a demand predictor that forecasts the expectation, volatility, and correlation of the streaming traffic associated with different videos based on statistical learning. Extensive simulations are conducted to evaluate our proposed algorithms, driven by the real-world workload traces collected from a commercial VoD system.