Toward Pattern-based Model Selection for Cloud Resource Forecasting

Cloud resource management solutions, such as autoscaling and overcommitment policies, often leverage robust prediction models to forecast future resource utilization at the task-, job- and machine-level. Such solutions maintain a collection of different models and at decision time select to use the model that provides the best performance, typically minimizing a cost function. In this paper, we explore a more generalizable model selection approach, based on the patterns of resource usage that are common across the tasks of a job. To learn such patterns, we train a collection of Long Short Term Memory (LSTM) neural networks, at the granularity of a job. During inference, we select which model to use to predict the resource usage of a given task via distance-based time series comparisons. Our experimentation with various time series data representations and similarity metrics reveals cases where even sophisticated approaches, such as dynamic time warping, lead to suboptimal model selection and as a result significantly lower prediction accuracy. Our analysis establishes the importance and impact of pattern-based model selection, and discusses relevant challenges, opportunities and future directions based on our findings.